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Abstract 

As special education enrollment for children with autism spectrum disorders (ASD) has increased, 
school-based programs and providers have been challenged to expand the scope and quality of 
services. Researchers and school-based providers are aligned in the goal of providing high-quality 
services to students with ASD, however current literature does not address how training and 
implementation needs may differ by the age of children served. The current study evaluates 
variability in teacher fidelity of Classroom Pivotal Response Teaching (CPRT), an evidence-based 
naturalistic behavioral intervention based on the principals of applied behavior analysis. Data 
included 479 individual video units collected from 101 teacher and 221 student participants. 
Videos were coded using behavioral coding definitions and student demographic information was 
collected from parents of participating children. Analyses explored differences in fidelity of CPRT 
by age of students. Results indicate a significant relationship between classroom type 
(preschool/elementary) and teachers’ fidelity of CPRT, suggesting the possibility of targeted 
training based on student age. 

Keywords: Autism spectrum disorder, Teacher training, Classroom pivotal response teaching, 
Intervention fidelity 


Introduction 

Autism spectrum disorder (ASD] is a neurodevelopmental disorder characterized by 
restricted or stereotyped patterns of behavior and impairments in social communication 
(American Psychiatric Association, 2013], Current estimates of the prevalence of ASD in 
the US are 1 in every 68 children, more than twice the rate since 2000 (CDC, 2014], Data 
collected by the National Survey of Children's Health (NSCH] indicate that the rate of 
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parent-reported ASD in school-age US children has increased significantly since 2007 
(Blumberg, et al., 2013], 

The prevalence of ASD highlights the need for increasing numbers of educators trained to 
teach students affected by the disorder. However, because ASD is a complex disorder and 
children with an ASD diagnosis show considerable variability in level of functioning and 
learning goals, adequately preparing educators is an on-going challenge. Standard teacher 
preparation programs offer only introductory courses in special education (Winn & 
Blanton, 2005], and even special education credential programs may not address the 
particular set of needs presented by students with ASD (Holdheide & Reschly, 2008], 
Indeed, research indicates practicing teachers who are credentialed and trained in broad 
special education practices may still lack some of the specialized expertise necessary to 
support students with ASD (Loiacono & Allen, 2008; Williams, Fan, & Goodman, 2010], A 
recent survey of nearly 200 universities offering teacher preparation programs in ASD 
found extreme variability in the type, content, and amount of training provided (Barnhill, 
Polloway, & Sumutka, 2010], Taken together these findings suggest teachers require 
supplementary training in best practices for educating students with ASD beyond their 
initial pre-service credentialing. 

Recent examination of practices frequently used by special education providers revealed 
that educational practices lacking in research support were used with equal or greater 
frequency by teachers compared to methodologies with a strong empirical base (Burns & 
Ysseldyke, 2009; Hess, Morrier, Heflin, & Ivey, 2008; Stahmer, Codings, & Palinkas, 2005], 
This is not due to a lack of evidence-based educational practices for youth with ASD. In 
fact, there have been several systematic reviews of the scientific literature focused on 
identification of the strategies or programs with the strongest evidence supporting their 
effectiveness for educating students with ASD (Odom, Collet-Klingenberg Rogers, & 
Hatton, 2010; National Autism Center, 2009 & 2015; Wong et al., 2015], The most recent 
and comprehensive systematic review identified 27 evidence-based practices (EBP] as 
effective for educating children with ASD (Wong et al., 2015], However, despite the 
identification of EBP for ASD, the dissemination and use of these practices in community 
programs is limited (Cook, Landrum, Tankersly, & Kauffman, 2003], These findings 
indicate a significant gap between current knowledge of EBP and the services students 
with ASD actually receive. This research to practice gap reflects a broader issue of 
disseminating research findings into usual care and has been identified across mental 
health and educational disciplines, generating recommendations for more effectively 
moving research into practice (e.g., Fixsen, Naoom, Blase, Friedman, & Wallace, 2005], 
These recommendations include improving training and support for practitioners’ use of 
EBP and evaluating factors that may influence EBP implementation. 

Several factors have been identified as related to teachers' use of EBP in school settings, 
including access to high-quality training and personal beliefs. Teachers demonstrate 
marked improvement in fidelity (or accurate use of the intervention as it was intended] 
after participating in EBP-focused trainings (Suhrheinrich, 2011; Suhrheinrich, Stahmer, 
Reed, Schreibman, Reisinger, & Mandell, 2013], However, receiving training alone is 
unlikely to sustain practice. Instead, teacher implementation is likely impacted by the 
type/intensity of training provided and access to on-going support within the classroom 
(Suhrheinrich, 2014], Kavale and colleagues (2000] noted that teachers' use of EBP tended 
to be influenced more by personal beliefs than research data. Teachers rated colleagues, 
workshops and other informal sources of information higher in terms of accessibility and 
trustworthiness compared to college courses and professional journals (Landrum, Cook, 
Tankersley, & Fitzgerald, 2007], Additionally, teachers report a primary motivation for 
using a specific strategy or practice is because they feel "it works" for their students 
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(Boardman et al., 2005; Landrum, Cook, Tankersley, & Fitzgerald, 2002; Slavin, 2002], it is 
developmentally appropriate for the students' age/functioning level (Stahmer, 
Suhrheinrich, Reed & Schreibman, 2012] or that there is a good fit between the practice 
and their classroom needs (Gersten, Woodward, & Morvant, 1992], A targeted survey of 
only providers who serve students with ASD identified similar patterns in program 
selection, with teachers choosing practices based on what they feel "worked for their 
students" or personal preference (Stahmer, Collings, & Palinkas, 2005], 

Age and Adaptation of Intervention Practices 

Teachers who report using EBP also often report individualizing or adapting those 
practices based on the needs and characteristics of their students. They selectively 
implement individual components of a practice based on their belief of how well the 
components fit with the needs of their classroom, rather than using the intervention as a 
whole (Stahmer, Suhrheinrich, Reed, & Schreibman, 2012], One particular student 
characteristic that significantly impacts teacher's perceptions and eventual practice 
selection is student age, with research indicating differences in teacher beliefs across 
student age/grade level. When asked about expectations for student behavior, ratings of 
student assertive behavior were more important for elementary and middle school 
teachers as opposed to high school teachers (Lane, Wehby, & Cooley, 2006], Furthermore, 
an evaluation of early childhood teachers (preschool through 3rd grade] indicated 
differences in reported and observed use of developmentally appropriate practices by 
grade level (Vartuli, 1999], In this study, teachers in lst-3rd grade classrooms did not rate 
developmentally appropriate practices as highly or use these approaches as much as 
preschool and kindergarten teachers. Other research has replicated these, or similar, 
findings of grade level differences, with teachers at preschool and lower elementary grade 
levels showing significantly higher ratings and practice of developmentally appropriate 
beliefs as compared to teachers of upper elementary grade levels (Buchanan, Burts, Bidner 
& White, 1998; File & Gullo, 2002], 

These differences in beliefs and practices across age ranges also highlight the possibility 
that student age may affect the goodness of fit between a given EBP for ASD and teacher 
use of the intervention in teachers' daily practice. The consideration of the EBP-student 
age fit is likely warranted, as EBP are currently supported for a specific age range, and/or 
may have varied efficacy for students of different ages (e.g., Wong et al., 2015], 
Examination of these variations may provide insight into how to best provide targeted 
training to teachers based on the age of students they support. In particular, given the 
influence of teachers' beliefs regarding the fit between the practice and student need, it is 
possible that tailoring or individualizing training to address perceptions of fit for EBPs 
may ultimately result in increased use. 

PRT and CPRT. Pivotal Response Training (PRT; Koegel, Schreibman, Good, Cerniglia & 
Murphy et al., 1989] has been identified as one EBP for children with ASD (National 
Autism Center, 2015; Odom, Collet-Klingenberg, Rogers, & Hatton, 2010], PRT is a 
naturalistic, developmental intervention based on the principles of applied behavior 
analysis that is considered evidence-based for children ages 0-14. Classroom Pivotal 
Response Teaching (CPRT] is an educational intervention for students with ASD that has 
been systematically adapted from PRT for use in school settings (Stahmer, Suhrheinrich, 
Reed, Schreibman & Bolduc, 2011], Preliminary data supports the efficacy of CPRT for 
educating students with ASD ages 3-10 in a classroom setting (Stahmer, Suhrheinrich, & 
Rieth, 2016], A recent randomized trial of CPRT in preschool and elementary classrooms 
offers a unique opportunity to examine how teacher implementation of this intervention 
varies across student age. This exploration may provide important insight into the 
appropriateness of strategies for certain ages of student as well as focused targets for 
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training based on student characteristics. The current study examines the influence of 
student age as measured by classroom type (preschool or elementary) on teachers' use of 
CPRT in special education settings. 

Methods 

The current study included a subset of data collected as part of a randomized waitlist 
control trial evaluating the effectiveness of CPRT in a large urban county in California. 
Teacher participants were randomly assigned to one of three CPRT training cohorts. Data 
for the current project include data collected during the year the teacher received CPRT 
training after all training was complete. 

Participants 

Participants included special education teachers (n=101) in a large urban county in 
California; 88% percent of teachers were female. Student participants (n=212) were ages 
3-12 and receiving special education programming under the category of autism. A 
majority of students were male (80%). Teacher and student participants were classified as 
either Preschool or Elementary based on teacher report and student age reported in 
demographics surveys (see Measures). See Table 1 for additional information on student 
and teacher demographics. 


Table 1. Participant Demographics 



Teachers 
n (%] 

Students 
n (%] 

Gender 

Female 

Male 

90(88] 

12(12] 

34(16] 

178(84] 

Ethnicity 

Hispanic or Latino 

18(18] 

78(36] 

Not Hispanic or Latino 

73(72] 

88(41] 

Unavailable 

10(10] 

47(23] 

Ethnicity 

African American 

0(0] 

13(6] 

Asian American/Pacific Islander 

3(3] 

10(5] 

Caucasian/White 

75(74] 

72(34] 

Filipino American 

2(2] 

1(1] 

Mixed 

3(3] 

18(8] 

Native American 

0(0] 

5(2] 

Other 

8(8] 

37(18] 

Unavailable or Not Reported 

10(10] 

56(26] 

Level of Education 

Bachelor's Degree 

38(38] 

- 

High School/GED 

1(1] 

- 

Master's Degree 

52(51] 

- 

Unavailable 

10 (10] 

- 

Age 

3-5 

- 

95 (45] 

6-8 

- 

72 (34] 

9-12 

- 

45 (21] 
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Procedure 

Project information was shared with all school districts in the county serving at least 15 
students under the classification of autism in preschool through 5th grades. Of the 24 
eligible districts, 17 districts agreed to participate. Special education directors were asked 
to provide information about the study to teachers in their own district who taught 
preschool through fifth grade classes with at least two students served for autism. Project 
staff met with interested teachers and consent to participate was collected. After 
enrollment in the study, each teacher was asked to select two target students for the 
project and parental consent to participate in the study was collected. Teachers were then 
randomly assigned to receive training in Year 1, 2, or 3. During the assigned training year, 
each teacher received 12 hours of CPRT training (i.e. lectures, small learning groups, 
discussion sessions and lesson planning] over the course of 6 sessions. Teachers also 
received 1:1 coaching in their own classroom with a CPRT trainer after the initial 6 hours 
of training. Coaching support continued throughout the training year at a rate of 
approximately one appointment every other week for each teacher. 

Measures 

Demographics Survey. Participant demographics were collected upon enrollment in the 
study via surveys completed by the teachers and parents. Participant completion of 
demographic measures was 94%. 

Classroom type. Classrooms were divided into Preschool and Elementary categories based 
on the teacher classification of his or her classroom (as reported in the demographics 
survey). The age range of 3 to 5 corresponded to the Preschool category and the range of 6 
to 12 corresponded to the Elementary category. 

Video Observations. Video data of each teacher working with his or her target students 
were collected after CPRT training was provided. Members of the research team visited 
the participating teacher's classroom to film observations. The video observations were 
conducted during a teacher-selected activity that remained constant throughout the 
school year. Recorded observations were typical classroom activities such as small group 
learning, circle time, and one on one instruction. The observations were designed to 
capture how/whether teachers had generalized the information they learned in the EBP 
training and were able to incorporate the EBP practices in the day-to-day running of their 
classroom. 

Fidelity of CPRT. Fidelity of CPRT was assessed based on video observation data. 
Behavioral definitions for 11 CPRT components and four associated skills were developed, 
and coders used a Likert Scale (1-5) to assign scores. A score of 4 or higher indicates 
meeting fidelity for that specific component of CPRT. Summary scores were calculated 
across four categories of FI items: maximizes student motivation, facilitates student 
responding, provides appropriate consequences, and prepares for session and manages 
environment (see Table 2 for a description of the content in each summary score). 
Research assistants who were blind to training condition were trained to a reliability 
criterion of 80% agreement across all components on two consecutive videos prior to 
coding independently. The trained coders evaluated 479 individual video observations. A 
second coder evaluated 35% of the total videos (n=171), and inter-rater reliability was 
assessed using intra-class correlation coefficients (ICCs). The ICCs were fair to good 
according to accepted standards (ICCs range: .60 - .81) (Cicchetti, 1994). 
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Table 2. CPRT Fidelity Summary Score Content 


Summary Score 

Description 

Maximizes Motivation 

The teacher employs a variety of strategies aimed at building student 
motivation, e.g. providing choices, taking turns and using preferred materials. 

Facilitates Responding 

The teacher obtains student attention and facilitates student responses via 
appropriate cueing with a variety of materials. 

Provides Consequences 

The teacher acknowledges and reinforces correct or approximately correct 
responses; the teacher also maintains student motivation with the rate of 
reinforcement provided. 

Prepares for Session 

The teacher structures the environment to promote educational goals by 
managing distractions and instructional materials; the teachers' uses varied 
prompting strategies. 


Analytic Plan. To examine differences in fidelity across classroom types, a linear mixed 
model analysis was utilized using the mixed procedure in SPSS 23.0 (SPSS, Inc., 2015], 
This approach was chosen given its ability to account for the nested study design (i.e., 
students nested within teachers) and presence of unbalanced data (e.g. non-normality, 
unequal variances) (Bolker, Brooks & Clark et al., 2008; Raudenbush & Bryk, 2002). 
Classroom type was specified as a fixed effects and participating teachers comprised the 
random effects. 

Results 

For measures Maximizes Motivation and Provides Consequences, teachers who taught in 
preschool classrooms had significantly higher fidelity of CPRT compared to teachers in 
elementary classrooms. In contrast, teachers who taught in elementary classrooms had 
significantly higher fidelity for Facilitates Responding than teachers who taught in 
preschool classrooms. Out of all summary score categories, teachers achieved highest 
mean fidelity on preparing for instructional sessions. No differences were identified in 
teacher FI for Prepares for Session. Full results are presented in Table 3. 


Table 3. Linear Mixed Effect Models examining differences across Classroom Type 


Fidelity Summary Component 

Classroom Type 


Preschool 

Elementary 

t 

Maximizes Motivation 

3.28 (0.83) 

3.03 (0.80) 

4.63* 

Facilitates Responding 

3.62 (0.69) 

3.72 (0.55) 

-2.71* 

Provides Consequences 

2.59 (1.05) 

2.12 (1.00) 

4.86* 

Prepares for Session 

4.11 (0.57) 

4.14 (0.48) 

-.61 

Average Fidelity across Components 

3.43 (0.47) 

3.27 (0.44) 



*p < .05; ** p < .01 


Discussion 

As the rate of diagnosis of ASD has increased, so has school enrollment of students served 
for autism. This has placed more demand on schools to provide high-quality educational 
services that are tailored to meet the needs of students with ASD. Identifying factors 
related to teachers’ effective use of EBP is a critical step toward improved programming 
for students with ASD. The results of this study provide several important outcomes 
relevant to increasing teacher use of EBP for ASD. 
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A primary question guiding the current study involved the relationship between 
classroom type/student age and teacher fidelity of CPRT. Previous research has explored 
how student factors, such as cognitive functioning or ASD severity, are related to student 
outcomes (e.g., Ben-Itzchak & Zachor, 2007; Harris & Handleman, 2000; Young, Falco & 
Hanita, 2016], However, there is no information examining how classroom type/student 
age may relate to teachers' ability to implement EBP. Our preliminary outcomes suggest 
that preschool teachers may have fewer challenges implementing a naturalistic, evidence- 
based intervention such as CPRT than teachers working with elementary age students. 

Increasing teacher use of EBP with high fidelity is a primary target of professional 
development. However, detailed information about teacher implementation has been 
limited in the scientific literature. In the current study, adequate fidelity was defined as a 
score above 4 on a 5 point Likert scale as rated by objective, trained observers. Teachers 
on average did not meet these criteria across three areas of intervention (antecedent, 
student responsivity, and consequence components]. Across both groups, results 
illustrated a pattern of low usage on average. Fidelity criteria were met on the Prepares 
for Session summary score, suggesting that teachers were effectively preparing for their 
teaching sessions and managing distractions in the classroom but struggled to use the 
specific EBP components consistently. 

These results address factors that serve to impact teacher's use and effectiveness of EBP. 
In particular, teachers in the current study received training that incorporated best- 
practices (e.g., ongoing coaching, performance-based measurement of skills; Stahmer et al., 
2016; Suhrheinrich & Chan, in press]; however, our data suggest that they struggled to 
implement the full CPRT intervention effectively in their classrooms. This may be due to 
several factors including the complex nature of autism interventions, the need for 
additional training or coaching, or variation in use of specific components based on fit 
with teaching strategies or student needs in the classroom. Incorporating further training 
in addition to teacher's other responsibilities may not be feasible due to time constraints 
and cost (National Education Commission on Time & Learning, 1994], The current 
findings point to a way to potentially individualize and target training. More specifically, 
results suggested that teachers' effective implementation varied as function of both 
specific components of the intervention and age of student served. Elementary teachers 
struggled to effectively maximize their student's motivation and to provide appropriate 
consequences, whereas preschool teachers demonstrated more difficulty obtaining 
student attention and facilitating responding. 

It is possible that specifically targeting training based on age of students served may 
promote more effective and consistent use of evidenced-based strategies and practices. 
For example, maximizing motivation and providing consequences (in this case direct 
reinforcement related to the activity] may be more feasible in a preschool environment 
where toys and other preferred materials are often used as teaching stimuli. In contrast, it 
may be easier to gain a student's attention and provide appropriate cuing in elementary 
classrooms where most students are conducting work at desks or tables and teachers may 
have a more structured environment. Therefore preschool teachers may require 
additional training and coaching in gaining a student's attention during large group (e.g., 
circle time] and less structured (e.g., free play with peers] activities. Elementary teacher 
training may focus on methods of providing tangible, direct reinforcement during 
structured activities. Further examination of the use of CPRT may help researchers 
determine whether all of the components are required in both settings. For example, it is 
possible that elementary school teachers are using longer-term reinforcement 
contingencies based on student's developmental understanding. If this is the case, and is 
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successful, fidelity and training requirements could be adapted to meet the needs of the 
specific classroom. 

Future work should examine the relationship between fidelity of CPRT and student 
outcomes across the different types of classrooms and age groups. This will help shape the 
level of fidelity necessary for facilitating student progress. There is early research to 
suggest that teachers' differential fidelity to specific components of a comprehensive 
treatment package is related to student outcomes, even if overall intervention fidelity is 
relatively low (Pellecchia et al., 2015], This suggests that an important next step in 
increasing the feasibility and effectiveness of EBP in community programs is to better 
understand which specific components of EBP lead to positive outcomes across different 
classroom types and ages. Given the heterogeneous needs of students with ASD, it is 
possible that different components of packaged EBP may be more or less influential for 
certain students, and vary by learning goal (Jobin, 2012], Therefore, a component-based 
approach to examining fidelity, such as the one used in the current analysis will provide 
information about the impact of teacher-initiated adaptation and individualization of 
interventions based on the needs of each specific classrooms serving students with ASD. 

This study provides preliminary data indicating a need to examine the use of EBP based on 
child and classroom characteristics. Future studies may consider classroom activities and 
format that are most conducive to specific EBP for children with ASD which may help 
teachers and schools chose appropriate interventions for different age groups and 
programs. This will, in turn, increase the relevance of training and decrease training time 
and expense for interventions that are not useful in specific settings. We recommend 
looking carefully at fidelity of EBP and what may affect teacher use of new strategies to 
help understand how to best facilitate effective training. 


• • • 
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